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What is a workflow? 



Workflows automate queries. 

One-time 

Standing 

Every search type can be a workflow. 

■ Same functionality and capability 

Follow on actions 

■ Email alert 
Download actions 

■ Metadata summary 
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Who can submit a workflow? 







Anyone! 

One owner per workflow 

Multiple-users can be notified 

If ownership needs to be changed, a ticket 
can be submitted to the team. 

Future: sharing workflows 

■ Right now, only the owner has the results in their 
“My Results” view. 
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What can I do with a workflow? 



r 




Workflows can be configured to run once 
Workflows can be configured to run daily 

Every 1 , 2, 3, 4, 6, 8, 12 or 24 hours 

■ You can set an offset to start running at a certain 
hour 

Download results 
Email results and email alerts 
MAILORDER results 
MySQL report 
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Why do I want a workflow? 



XKEYSCORE has a rolling buffer of data 
Repetitive queries 
Sigdev purpose 

Fingerprint and appid testing 

Queries take a long time during high times 
Follow on actions 

■ Google Earth data 

■ Statistics 

Customizable - write a script! 
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How do I setup a workflow? 




« 



Navigation Menu 
d _j Explorer 
fcl Home 

d Central 

: f] Request 
if] My Workflows 
d 'z3 Search 
d ^3 Classic 

±) ,J MuttiSearch 
+) O Classic A-M 
d fj Classic N-Z 
d_d Common 

|f] Category DNI 
f] Document Metadata 
Email Addresses 



XfCEYSCGRE Welcome: 
Preferences M* Kelp 



switch users 



l±) O User Activity 
fl QVolP 
G3 C3 Wlrefess 
d PlRasutts 

(f] My Recert Results 
fel My Previous Reauls 
jgj My Ongoing Results 
(f]My Downloads 
3 0J Statistics 

bnk Sunmareatlon 
3£)Taggng 

fel Local Tagging 

1^1 Yflnninn 



Welcome to the /Veit* XKEYSCORE Home Page! 

If you have questions or bug reports please go to X KEYS CORE New GUI Forum 
To use the old GUI, click here 

HUMAN RIGHTS ACT, 

USSID 18 AND USSID 9 

I (SYSTEM) queries require a justification to ensure Human Rights Act (HRA), USSID 18 and 
55ID 9 compliance, Please enter information as prompted by the query interface. An audit 
ail has been established and will be searched as part of Menwith Hill station's response to 
iy complaint brought under hra and as part of the USSID ie and USSID 9 process . 
ease note that sensitive Targeting approval (sta) is required for hra before submitting 
iy query which includes terms specific to a person or company (eg name, address, identity 
atails such as communications address, passport/bank account number) who EITHER (a) is 
afined as a UK, British Dependent Territory (BDT) or Second Party "person" or (b) is located in 
ie UK, or a BDT or Second Party country. STA Is also required for wildcard pulls that are 
evitabfy going to retnve a substantial proportion of such enties (e.g. wildcarding on a UK city 
ide). Full legal guidance is available from the HRA Compliance Officer at Menwith Hill Station. 
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How do I setup a workflow? 
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How do I setup a workflow? 
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Workflow Central Request Wizard 
Bask Information 



X 



Query Name: 

Query justification: 
Additional Justification: 
Miranda Number: 


Find_my_appid 




Testing appid signature 


- 




Datetime; 1 Day jjstart: 2009-03-04 E 00:00 £stop: 2009-03-05 □ 23:59 £ W 


Reccurring Search One Time 5earcfrj^ 


Basic Features Help \ v 



Runs once over 
a set datetime 
range 



Cancel 4 Prev 



IMext 



ring or one 

ist be unique per user 
must have a justification 
justifications 
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How do I setup a workflow? 
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Selec 

searc 



Select a 
field to 
search 



Workflow Central Request Wizard 



Add Search Fields 



Search Values are AlMDed by default 

To OR Search Fields: 

* Use the Multiple Field Search tab (below the input fields). 

* Select all the fields you wish to search. 

To DR Search Values: 

* Type “OR 1 between each value (no quotes). 

See Search Value Help below for more details or 
for a description of boolean logic go to here . 



Search Field 


Search Value 


Remove 


From IP Address OR To IP Address 




1 .2.3.4 


X 


Attribute Info 
From IP Address 




3 






To IP Address 








liFrom Port ■ 






[To Port 




z} 






Single Field Search 


Multiple Field Search 





Search Value Help 



X 



Cancel 4 Prev 



'? Next 




ant to 



or every field, 
du must select 
le PLUS key 
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Group by option 



Group b 
Red 
Reti 



Workflow Central Request Wizard 



Group Search Fields 



Would you like to group any Fields? 
NO 



Group By Type 



Table Unique Values: 

Global Unique Values: 
Columns to Group By 

Datetime: 

Client IP (X-Fowarded-For): 
Username: 

Attribute Info: 

From IP Address: 

To IP Address: 

From Port: 

To Port: 

From Country (IP): 

To Country (IP): 

From City (IP): 

To City (IP): 

From Latitude (IP): 



n 

r 

□ 

r 

D 

r 

n 

r 

n 

n 

r 

n 

r 



Cancel 4 Prey 







Group By Type Help 



This option groups paoh 
raetdtfeitel3tiabAfl^iTeSil3f L E and 
□vstaattebaettatel ihrssettu its . 
concatenated. 



Select the fields you 
want to group by. 



ita 




b Next 




results. 
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Follow on Actions 



All< 

Allc 

loca 

All< 



Workflow Central Request Wizard 



Follow-on Actions 



X 








Would you like to add any follow on actions 


^ No 
yes 


Script 


Script Arguments 


Actd 




Email To: 




Email Alert v 




RGWR; r Return Only With Results 


Email Alert 




SQL Report 






Download Sessions 





intent) to another 






Cancel 4 Prey 



^ Next 
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Email alert 



r f 



Workflow Central Request Wizard 
Follow-on Actions 



Would you like to add any follow on actions 



r NO 
Yes 



Script 



Email Alert 



Cancel 4 Prev 



Script Arguments 



Add 



Email To: 



ROWR: 



1“ Return Only With Results 




Comma delimited email 
addresses. 

This option only sends an 
email if you workflow has 
results. 



> Next 
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SQL report 



r 



Workflow Central Request Wizard 



Follow-on Actions 



Would you like to add any follow on actions 



r No 

- 

& Ye; 



Script 



SQL Report 



Cancel 'i Prev 



Script Arguments 



Add 



Type: 

Email To: 

Email Subject: 
Email Content: 
Email 

Attachment: 

ROWR: 

Filename: 

Mail Order 
Trigraph: 

SQL: 



G2IP: 



P Email Attachment 
r Return Only With Results 




SELECT 

FROM %{0UTPUT_T ABLE} 

WHERE 

GROUP BY 



d 



f~ Compress Contents 



> Next 




CSV or HTML 



Th i sit i us t be a VAL I D 3QL 

sfeM^ tadata that a user 

can set. 

'Example. 

SELECT casenotation, sigad 
FROM %{OUTPUT_TABLE} 
WHERE sigad!=‘ ! 

GROUP BY casenotation 
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Download Results 






Workflow Central Request Wizard 



Follow-on Actions 



Cancel 4 Prev 









WQUia you iik ei to aou any ronow on actions 


r MO 






c ' Yes 






Script 


Script Arguments 




Add 




User ID: 








Download Sessions 


V 






4- ( 






Email To: 










Email Subject: 










Email Content: 








ROWR: 


r Return Only With Results 






Filename: 










Mail Order 








Trigraph: 








GZIP: 


r Compress Contents 






Send To Agility: 


r Send To Agility 





k Next 
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Workflow Central Request Wizard 



X 



Workflow Review 

This query (Find_my_appid) will search the Full Log table in database (s); 

xks-jychan:qO 

The query will run COINITINUGUSLV executing every 6 hours beginning at 5;Q0 EST 
The query will execute the following search criteria; 



<and> 

<fie!d>From IP Address </fi eld > 
cvalue >1.23. ■ 4 </ value> 

</and> 

<and> 

<fie!d>To Port</field> 

<value >80 </ value > 

</and> 

cand> 

<field>AppID C+Fingerprints) + </field> 
<value >search/ goo gle*</ value > 
</and> 



Workflow Values [j Workflow XML 




Submit 
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Workflow Pending 



XKEYSCORE 

Home 'ff' Wariflcv* Central Search Results Statistics [^| Tagging Preferences W He^ 

- My Workflows 
Help Actions T 



welcome; jychan 5V*M, BWS, 



Navigation Menu 
^^Jtsplcirer 
^Home 

Q&VlfcrkfkJW Central 
13 Request 
J=]My Woittas 
3 £3 Search 
U-£j]Gasslc 

3 f~lt'.'[JtiSeerch 
Classic A-M 
itiaassic M -1 

3 0 Common 

H category DM 
fc] Document MstecJeta 
0Eznai Addresses 
jE] Exacted Files 
t]Ful Lag DNI 
HTTP Activity 

HlftionePJumloer Extractor 
5 i]UsBr Acth/ly 
3 0 Dictionary MBs 
3 QF4e Transfer 
0 0Muli£eerDh 

^IP Addresses 
Address 
Username 

ij 0 Network Management 
^Searcfi Wizard 
3 0 LIscrAc-tivrty 

aCivoF 

Q 0 Wireless 
3 0 Results 

^]My Recent Results 
gMy Previous Results 
H My ongoing Resorts 
^]Mv Dow ni cads 
3 0] Statistics 

HLinfcSummarizailon 

d0Taggrg 

=jLpcgl Toggng 



Query Type 
(jj fulljog 



Query Name 
Frd_my_appid 



Last Modified 
2009-03-0514: 44:' 5 



State * 
pending 



Actions 

'*©> tr 



State * 


Actions 


pending 


xu. n 



t| Page P of ! 



it h 



Page Siaec 30 



Displaying 3 - 2 of t 



This system is Audited tor USSID IS And Human R.iqh(;s Ac 


H 


cornpliincH 


TOP SECRET//COM)rST//REL TO USA, All's C AM. GBR. .111 


<1 


\'?L//20320 10B 
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Workflow Approved 





This system 


is au< 


Jitad for USSCD IS and Human Rights Act compliance 


TOP SECRET/ /COMI NITj 


u 


TO llf»A, AUS, CAR;, <;BR. and: N7i.//2fl3201ftF 



XKEYSCORE 

Home ©" Workfbw Central \ Search , Results l_J Statistics |_J Tagging Preferences Help 



Welcome: jy chan switch users 



Navigation Menu 

_l l' : ^ Explorer 

^1 Horne 

id”) Workflow Central 
i§3 Request 
jig] My Wnrkfbws 

□ -Ell Search 
30 Classic 

±J .. J MultiSearch 
±j Cj Classic A-M 
±j (J Classic N-z 
3 Common 

sj Cstegory DWI 
Document Metadata 
^ Emal Addresses 
^ EMraded Files 
Fril Lorj DNI 
=3 HTTP Activity 

Phone Number Extractor 
^ User Activty 
■3 P"! Dictionary Hts 
3 flFfe Transfer 
30 MultiSearch 

=3 P Addresses 
=~1 Mac Adc^ess 
tg Username 

3 £J Network Management 
g] Search Wizard 
d f_J User Activity 
it Dvoip 
(3 CDWfirrless 

□ 0 Re3kits 

j=t]!vty Recent Restfts 
^)Mv Previous Pfe3uts 
Hj]My Ongoing Results 
j^jMy D&wnbads 
3 0 Statistics 

jj^Lhlr Summarization 
dtiSTaponp 

i^jLocal Tagging 

T>(*h Fvfrdrtnr TWhrrhfi 



My wurkfltiws 



Het? Actions T 

Query Type 
_+i full Jog 



Workflow; Find_my_appid 






<?mi version ="1,0"' encoding ="y,IF-8“?> 

<queiyJobs> 

<inbernal_gui> l</infcernal_gui> 
<datetjme_created>1236264295</datetime_created> 

<iob> 

1 1 in hi I i | iii m I 

<xb / xkSJJSQrjname > 

odes p assword> 18B37b706 12 i«Dca < /xks_password > 

<search_type > fiJ Jog < /search Jtype > 

<query_name >Find_my_appid </query_name > 

<query Justification > Testing appid signature </query Justification 
<datetjm.e> 

< interval > &</interval > 

< offset >5</offset> 

</datetime> 

<sgl> 

< where > 

<and> 

<field>fmjp< /field > 

< value> 1,2 , 3 , 4 < /value> 

</and> 

<and> 

< field>to_ap < /field > 

< value>80 < /value > 

</and> 

<and> 

< field > fingerprint < /fiefd> 

< value>search/ggogle*</value > 

</and> 

</where> 

< group Jay > to Jp < /group Joy > 

< indexes > unique key (to Jp )</indexes > 

</sql> 

<advanced> 

< content_mu5t_exist > true </oontent_nnust_exist > 

< routing > 

< database>xks-jychan : qQ< /database > 

</routing> 

■ T ■ ■! . i ■ ii ■ ■ ■) ■ 



zi 



iipftiVHkard 



Caned 



Save/Submit 



4 
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Displaying 1 - i of 1 



This system as audited for USS[D IS and Human Rights Act compliance 



201DB 
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Common mistakes 



r /^ 



From IP and To IP 
with the same value. 

In this view, terms are 
ANDed together. 

Use Multiple Field 
Search Tab. 




Workflow Central Request Wizard 



Add Search Fields 



Search Values are ANDed by default. 

To OR Search Fields: 

* Use tine Multiple Field Search tab (below tine input fields). 

* Select ail Hie fields you wish to search. 

To OR Search Values: 

* Type 'OR* between each value (no quotes). 

See Search Value Help below for more details or 
for a description of boolean logic go to here . 



Sps'^c!'. r icjij 

From IP Address OR To IP Address 



Search Value 
1 .2.3.4 



Remove 

n 



Attribute a mo 
From IP Address 
To IP Add ress 



From Port 



To Port 



31 



Single Field Search Multiple Field Search 



Search Value Help 



Cancel 4 Prev 



> Next 
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Common mistakes 






Using the multiple 



Workflow Central Request Wizard 




field search does not 
break this up into 3 
search<->value pairs. 



Search Values are AlMDed by default, 

To OR Search Fields: 

* Use the Multiple Field Search tab (below the input fields). 

* Select all the fields you wish to search. 

To OR Search Values: 

* Type 'OR' between each value (no quotes), 



Enter each term 
separately in the 
singe fieldsearch. 



Search Field 
From IP Address 
To IP Address 

From Port 

i 

Single Field Search 
Search Value Help 



See Search Value Help below for more details or 
for a description of boolean logic go to here , 

Search Value 
1 . 2 . 3.4 
5 . 67.0 
80 



Multiple Field Search 




Cancel 4 Prev 



k Next 
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Common mistakes 



r 



This will return ALL 
casenotations. 

a will be deafeted 
by “!a” but a does 
equal “!b” 

All the defeated 
values must be 
ANDed together. 




Workflow Central Request Wizard 



X 



Add Search Fields 



Search Values are ANDed by default. 

To OR Search Fields: 

* Use the Multiple Field Search tab (below the input fields). 

* Select all the fields you wish to search, 

To OR Search Values: 

* Type 'OR' between each value (no quotes), 

See Search Value Help below for mere details or 
for a description of boolean logic go to here , 



Search Field 


Search Value Remove 


Casenctation 

Casenctation 


!a x 

<b x 


Casenotation 


fc X 


Casencrtation 


!d X 


L_ u 


v SI ' 


Single Field Search Multiple Field Search 




seartn vaiue rteip 


|T| 



Cancel 4 Prev 



> Next 
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Common mistakes 



Workflow Central Request Wizard 
Add Search Fields 

Search Values are AISIDed by default. 

To OR Search Fields: 

* Use the Multiple Field Search tab (below the input fields), 

* Select all the fields you wish Id search, 

To OR Search Values: 

* Type 'OR 1 between each value (no quotes). 

See Search Value Help below for more details or 
for a description of boolean logic go to here . 



x 



Search Field 
Casenotdion 
Casenotortion 
SIGAD 



Search Value 

!c 

Id 

AUC-S93 



Remove 

x 

at 

X 

GL 



Cana 



Select the Database(s) to query 



Basic Features Help 




If you are selecting 
specific SIGADs, only 
select the sites that 
have data from that 
SIGAD. 

Queries will return 
faster 

Slrig tet€ftGAI3cted 
c Less work for the 

system. 



AUS sites 
|v F6 sites 
1^ NZ sites 

Content must exist 






0 j Check fill; 




■ 1 Uncheck All 
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Common mistakes 







If you select the 
SQL Report option, 
make sure you put a 
valid SQL statement! 



SQL statement filled in: 

SELECT casenotation, 
courfiftflpTY 

fr<^^o??50Vput_table} 

WHERE casenotation !=" 
GROUP BY casenotation 



Workfl o w Cen tra I Req nest Wizard X 



Follow-on Actions 



Would you like to add any follow on actions 

r No 
& Yes 



Script 


Script Arguments 


Add 




Type: 


CSV 




SQL Report 




+ 














Email To: 


analyst^iwork.com 










Email Subject: 


My Workflow Results 










Email Content: 


Bad SQL - empty 










Email 

Attachment: 


r Email Attachment 










ROWR: 


r Return Only With Results 










Filename: 












Mail Order 


1 








Trigraph: 












SQL: 


SELECT casenotationj count (*) 
FROM %{OUTPUT_T ABLE} 

WHERE casnenotation!-’ 1 
GROUP BY casenotationl Cll 










GZIP; 


COiuprCSS rnntpntf 







Cancel 4 Prev ^ Next Submit 
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Questions? 

xks_workflow@r1 .r.nsa 




B 
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